Improving selection stability of multiple testing procedures for fMRI

In search of an appropriate thresholding technique in the analysis of functional MRI-data, several methods to prevent an inflation of false positives have been proposed. Two popular (voxelwise) methods are the Bonferroni procedure (BF), which controls the familywise error rate (FWER), and the Benjamini-Hochberg procedure (BH), which controls the false discovery rate (FDR) (Benjamini & Hochberg 1995). Multiple testing procedures are typically evaluated on their average performance with respect to error rates, ignoring the aspect of variability. Resampling techniques allow to assess the selection variability of individual features (voxels). Following the approach of Gordon, Chen, Glazko & Yakovlev (2009) in the context of gene selection, we investigated whether variability on test results for BF and BH can be reduced by including both the significance and selection variability of the voxels in the decision criterion

Adaptive thresholding for fMRI data

In the analysis of functional MRI-data, several thresholding procedures are available to account for the huge number of volume units or features that are tested simultaneously. The main focus of these methods is to prevent an inflation of false positives. However, this comes with a serious decrease in power and therefore leads to a problematic imbalance between type I and type II errors (Lieberman & Cunningham, 2009). In this research, we present a method to estimate the number of activated features. The goal is twofold: • Given the expected number of active units, widely used methods to control the false discovery rate (FDR) can be made adaptive and more powerful. • The type I and type II error rate following such a thresholding technique can be estimated enabling a direct trade-off between sensitivity and specificity. Chen, Wang, Eberly, Caffo, & Schwartz (2009) argue that activation foci in fMRI data are often small and local leading to a large proportion of null voxels. However, task-related activation is expected to occur in clusters of voxels rather than in isolated single voxels. We consider peaks of activation instead of voxels and provide a procedure to estimate the number of active peaks. Concentrating on peaks leads to an enormous data reduction, and the proportion of non-null hypotheses can be expected to be much larger among peaks than among voxels. Given an estimate of the number of active and non-active peaks, we demonstrate how an adaptive FDR controlling procedure on peaks can be obtained and how false positive and negative rates associated with this procedure can be estimated. This allows researchers to reconsider the balance between sensitivity and specificity in function of study goals. The method is evaluated and illustrated using simulation studies and a real data example. References Chen, S., Wang, C., Eberly, L., Caffo, B., & Schwartz, B. (2009). Addaptive control of the false discovery rate in voxel-based morphometry. Human Brain Mapping , 30 , 2304-2311. Lieberman, M. D., & Cunningham, W. A. (2009). Type i and type ii error concerns in fmri research; re-balancing the scale. Social cognitive and affective neuroscience, 4 , 423-428

Stability based testing for the analysis of fMRI data

Neurological imaging has become increasingly important in the field of psychological research. The leading technique is functional magnetic resonance imaging (fMRI), in which a correlate of the oxygen-level in the blood is measured (the BOLD-signal). In an fMRI-experiment, a time series of brain images is taken while participants perform a certain task. By comparing different conditions, the task-related areas in the brain can be localised. An fMRI study leads to enormous amounts of data. To analyse the data adequately, the brain images are devided into a large number of volume units (or voxels). Subsequently, a time series of the measured signal is modelled voxelwise as a linear combination of different signal components, after which an indication of activation can be tested in each voxel. This encompasses an enormous number of simultaneous statistical tests (+/-250 000 voxels). As a result, the multiple testing problem is a serious challenge for the analysis of fMRI data. In this context, classical multiple testing procedures such as Bonferroni and Benjamini-Hochberg (Benjamini & Hochberg, 1995) have been applied to respectively control the family-wise error rate (FWER) and the false discovery rate FDR)(Genovese, Lazar, & Nichols, 2002). Random Field Theory (Worsley, Evans, Marrett, & Neelin, 1992) controls the FWER while accounting for the spatial character of the data. Because of the dramatically decrease in power when controlling the FWER, methods to control the topological false discovery rate (FDR) were developed (Chumbley & Friston, 2009; Heller, Stanley, Yekutieli, Rubin, & Benjamini, 2006). A general shortcoming of current procedures is the focus on detecting non-null activation while a non-null effect is not necessarily biologically relevant. Moreover, failing to reject the hypothesis of no activation is not the same as confidently excluding important effects. Another aspect that remains largely unexplored is the stability of test results which can be defined as selection variability of individual voxels (Qiu, Xiao, Gordon, & Yakovlev, 2006). Given the need to control both false positives (type I errors) and false negatives (type II errors) in a direct manner (Lieberman & Cunningham, 2009), we approach the multiple testing problem from a different angle. Following the procedure of (Gordon, Chen, Glazko, & Yakovlev, 2009) in the context of gene selection, we present a statistical method to detect brain activation that not only includes information on false positives, but also on power and stability. The method uses bootstrap resampling to extract information on stability and uses this information to detect the most reliable voxels in relation to the experiment. The findings indicate that the method can improve stability of procedures and allows a direct trade-off between type I and type II errors. In this particular setting, it is shown how the proposed method enables researchers to adapt classical procedures while improving their stability. The method is evaluated and illustrated using simulation studies and a real data example

Alternative-based thresholding: a simulation study

See attachmen

Introducing alternative-based thresholding for defining functional regions of interest in fMRI

In fMRI research, one often aims to examine activation in specific functional regions of interest (fROIs). Current statistical methods tend to localize fROIs inconsistently, focusing on avoiding detection of false activation. Not missing true activation is however equally important in this context. In this study, we explored the potential of an alternative-based thresholding (ABT) procedure, where evidence against the null hypothesis of no effect and evidence against a prespecified alternative hypothesis is measured to control both false positives and false negatives directly. The procedure was validated in the context of localizer tasks on simulated brain images and using a real data set of 100 runs per subject. Voxels categorized as active with ABT can be confidently included in the definition of the fROI, while inactive voxels can be confidently excluded. Additionally, the ABT method complements classic null hypothesis significance testing with valuable information by making a distinction between voxels that show evidence against both the null and alternative and voxels for which the alternative hypothesis cannot be rejected despite lack of evidence against the null

neuRosim: An R Package for Generating fMRI Data

Studies that validate statistical methods for functional magnetic resonance imaging (fMRI) data often use simulated data to ensure that the ground truth is known. However, simulated fMRI data are almost always generated using in-house procedures because a well-accepted simulation method is lacking. In this article we describe the R package neuRosim, which is a collection of data generation functions for neuroimaging data. We will demonstrate the possibilities to generate data from simple time series to complete 4D images and the possibilities for the user to create her own data generation method

Searching for Imaging Biomarkers of Psychotic Dysconnectivity

Background: Progress in precision psychiatry is predicated on identifying reliable individual-level diagnostic biomarkers. For psychosis, measures of structural and functional connectivity could be promising biomarkers given consistent reports of dysconnectivity across psychotic disorders using magnetic resonance imaging. Methods: We leveraged data from four independent cohorts of patients with psychosis and control subjects with observations from approximately 800 individuals. We used group-level analyses and two supervised machine learning algorithms (support vector machines and ridge regression) to test within-, between-, and across-sample classification performance of white matter and resting-state connectivity metrics. Results: Although we replicated group-level differences in brain connectivity, individual-level classification was suboptimal. Classification performance within samples was variable across folds (highest area under the curve [AUC] range = 0.30) and across datasets (average support vector machine AUC range = 0.50; average ridge regression AUC range = 0.18). Classification performance between samples was similarly variable or resulted in AUC values of approximately 0.65, indicating a lack of model generalizability. Furthermore, collapsing across samples (resting-state functional magnetic resonance imaging, N = 888; diffusion tensor imaging, N = 860) did not improve model performance (maximal AUC = 0.67). Ridge regression models generally outperformed support vector machine models, although classification performance was still suboptimal in terms of clinical relevance. Adjusting for demographic covariates did not greatly affect results. Conclusions: Connectivity measures were not suitable as diagnostic biomarkers for psychosis as assessed in this study. Our results do not negate that other approaches may be more successful, although it is clear that a systematic approach to individual-level classification with large independent validation samples is necessary to properly vet neuroimaging features as diagnostic biomarkers

Associations of cannabis use disorder with cognition, brain structure, and brain function in African Americans

Although previous studies have highlighted associations of cannabis use with cognition and brain morphometry, critical questions remain with regard to the association between cannabis use and brain structural and functional connectivity. In a cross-sectional community sample of 205 African Americans (age 18–70) we tested for associations of cannabis use disorder (CUD, n = 57) with multi-domain cognitive measures and structural, diffusion, and resting state brain-imaging phenotypes. Post hoc model evidence was computed with Bayes factors (BF) and posterior probabilities of association (PPA) to account for multiple testing. General cognitive functioning, verbal intelligence, verbal memory, working memory, and motor speed were lower in the CUD group compared with nonusers (p \u3c .011; 1.9 \u3c BF \u3c 3,217). CUD was associated with altered functional connectivity in a network comprising the motor-hand region in the superior parietal gyri and the anterior insula (p \u3c .04). These differences were not explained by alcohol, other drug use, or education. No associations with CUD were observed in cortical thickness, cortical surface area, subcortical or cerebellar volumes (0.12 \u3c BF \u3c 1.5), or graph-theoretical metrics of resting state connectivity (PPA \u3c 0.01). In a large sample collected irrespective of cannabis used to minimize recruitment bias, we confirm the literature on poorer cognitive functioning in CUD, and an absence of volumetric brain differences between CUD and non-CUD. We did not find evidence for or against a disruption of structural connectivity, whereas we did find localized resting state functional dysconnectivity in CUD. There was sufficient proof, however, that organization of functional connectivity as determined via graph metrics does not differ between CUD and non-user group

Plan der Gränzen des Burgfriedens von Regensburg

Digital media availability has surged over the past decade. Because of a lack of comprehensive measurement tools, this rapid growth in access to digital media is accompanied by a scarcity of research examining the family media context and sociocognitive outcomes. There is also little cross-cultural research in families with young children. Modern media are mobile, interactive, and often short in duration, making them difficult to remember when caregivers respond to surveys about media use. The Comprehensive Assessment of Family Media Exposure (CAFE) Consortium has developed a novel tool to measure household media use through a web-based questionnaire, time-use diary, and passive-sensing app installed on family mobile devices. The goal of developing a comprehensive assessment of family media exposure was to take into account the contextual factors of media use and improve upon the limitations of existing self-report measures, while creating a consistent, scalable, and cost-effective tool. The CAFE tool captures the content and context of early media exposure and addresses the limitations of prior media measurement approaches. Preliminary data collected using this measure have been integrated into a shared visualization platform. In this perspective article, we take a tools-of-the-trade approach (Oakes, 2010) to describe four challenges associated with measuring household media exposure in families with young children: measuring attitudes and practices; capturing content and context; measuring short bursts of mobile device usage; and integrating data to capture the complexity of household media usage. We illustrate how each of these challenges can be addressed with preliminary data collected with the CAFE tool and visualized on our dashboard. We conclude with future directions including plans to test reliability, validity, and generalizability of these measures